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The objective of this study was to investigate a pro- 
posed method of auditing cost-effectiveness analyses of 
Mechnolog1ealfehengesithe method 1s based on the use of cost 
functions to predict costs of operating with and without the 
improved technology. A comparison of these predicted costs 
indicates whether the technological change is cost-effective. 
Results of the comparison can then be used to audit more 
detailed analyses of the change. The investigation employed 
econometric techniques in a case study application of the 
method to a technological change which occurred at the Naval 


Du cworsebacility c North Island, San Diego, California. 
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٣۰۰۰۰۰۰٠۱٠٦ ENTIT ٣۷ ان‎ OR THE PROBLEM 

tt iS commonly accepted that technological advancements 
are arriving at a faster and faster pace with the state of 
the art generally far ahead of the functional application. 
One reason for this gap is the costly enterprise of transfer- 
ing theory into practice. When the initial transfer is made 
a natural question to ask prior to extensive implementation 
is "Will it be cost-effective?" Current methods of cost- 
effectiveness analysis are tedious and usually employ a 
detailed analysis of the process which incorporated the 
change. Such analyses are inherently imprecise due to the 
possibilities of ما سط‎ dependencies, double counting 
of costs, and external changes which may affect the effective- 
ness of the process and yet not be accounted for in the 
analysis. Thus when a cost-effectiveness analysis is com- 
Pilot Ad a reviewing agency might well ask, "Are the conclusions 
of the analysis correct?! As mentioned, the methods em- 
ployed in the analysis make this second question a hard one 
LONAS WEN 

This study is solely concerned with investigating a 
method which may simplify the task of answering the second 


question; that is, a method for auditing cost-effectiveness 





—. 


analyses of technological changes. The investigation was 
accomplished by applying the method to a specific technolo- 
gical change which occurred at the Naval Air Rework Facility 
auslesen Island Mecaliformia (NARENI). While an indirect 
product of this study is a statement concerning the cost- 
effectiveness of the change which occurred at NARFNI, it is 
emphasized that the purpose of the study is to investigate 
and refine an auditing method initially proposed by Robert 
L. Spooner Re. D which was based on the estimation of 
production functions through linear least-squares regression 
techniques. A description of Spooner's proposal and later 
refinements established by Charles W. Bradley ucc and 


Wilber C. Trafton /Ref. 6/ will be given in Chapter II. 


B. NARFNI 

NARFNI is one of seven naval air rework facilities 
throughout the continental United States responsible for 
major maintenance, conversion and repair of United States 
Navy and Marine Corps aircraft and related components. To 
accomplish this mission NARFNI employes approximately 6800 
Civilian workers and spends $150 million on annual operating 
expenses /Bradley, pp. 8 and Le 

Maintenance of equipment is accomplished under one of 
three primary programs, with two cf the programs receiving 
inputs either directly from the customer or from the third 


program (see Fig. 1). 
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FIGURE 1. NARFNI Repair Programs 


Under these programs work is further assigned to one of 
the six divisions of the Production Department identified in 


Figure 2. 


Weapons £1 

Weapons £2 
Hydrolic/Mechanical 
Tower 7 

Avionics 

Components and Metal 


FIGURE 2. Production Department Divisions 


The assignment of work to various programs and production 
5395-1 o besthoucht of as a "Job shop! operation in 
which each shop performs a specific task. Thus an item 
entering rework is subject to being dismantled, its sub- 
dssembisosmdertributed tor rework, and then reassembled prior 
fo Feiturminogto the customer. This type of operationgplaces 
emphasis on the scheduling of rework in that ideally the 
shops are completing work on related parts: in the same se- 


quer oO educe ithe total repair time of thefmajor end items. 
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Scheduling is also important to the shop managers who must 
order A and plan personnel assignments based on the 
projected arrival of work from other shops. This entire 
schedulina problem is intimately related to the technological 
change to be described in the following section. 

NARFNI is primarily funded by the Navy Industrial Fund 
(NIF) which provides a working capital fund to finance repair 
operations. The finished product is returned with a bill 
to the customer for the work done. This "debt" 1s then paid 
by transfering funds from the customer's appropriated main- 
tenance funds to the NARF's working capital fund. Under this 
concept NARFNI is required to control its finances to incur 
zero profit at the end of each fiscal year. This control is 
exercised at Quarterly Planning Conferences in which repre- 
sentatives from the customers and NARFNI meet to plan the 
following quarter's work load input and prices for the work 
to be accomplished. 

The work load is measured in number of required man 
hours (NORM) of direct labor needed to accomplish the work 
requested by the customers. The NORM is estimated from 
historical data maintained by NARFNI on each type of work 
it has accomplished in the past. 

NARFNI is reimbursed under basically two types of con- 


tracts, cost reimbursable and fixed price, the majority of 


TO 
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the work being performed under fixed price contracts estab- 
lished at the Quarterly Planning Conference. This type of 
contract gives NARFNI incentive to minimize costs subject 

to the required work load, a fact which is instrumental to 


the auditing method investigated. 


C. WORK IN PROCESS INVENTORY CONTROL SYSTEM (WIPICS) 

In January, 1972, NARFNI installed an industrial informa- 
tion system to assist in the location and scheduling of 
inventory in the rework process. WIPICS was requested and 
initiated by the Management N Development Office (MSDO) 
in an effort to assist the NARF's in performing their ICT 
at a minimum cost to the government. If the prototype at 
NARFNI is determined to be cost-effective WIPICS will then 
be installed at the remaining six NARF's. Its major milestons 


are listed in Figure 3, ` 


Early 1969 - ROHR Corporation studied NARFNI 

Late 1969 - MSDO requested contract 

‘Early 1970 - NAVAIRSYSCOM evaluation and approval 
July 1970 - SECNAV authority for prototype 

Late 1970 - Contract negotiated 


4 January 1971 - Contract date, D-Day 

D+ 9.5 months - WIPICS developed 

PERII months WIPICS test 

D+ 11.5 months - WIPICS prototype started 


0-0 EO TE 5 ピロ 


PIGURD oe Wir tes Waites tones 
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Wir iG@omeonsistsS of ayeentral computer which is) linked 
to the job shops by 164 "touch tone" telephones, 20 alpha- 
numeric terminals, and four teletypewriters. In addition, 
the system is equipped with an 80 word audio-response unit 
to allow two-way communication through the telephones. When 
an item first arrives at NARFNI it is broken down into 
identifiable subassemblies. For each of these subassemblies 
a computer record is created and entered into WIPICS. This 
record contains information such as identification, required 
work, required material, location, and status. Each job 
shop inputs arrival and departure information through the 
"touch-tone" telephones. Also, an unexpected delay in work 
due to lack of material or some other shortcoming is inputed 
as "it “occurs.” At the completion of the work day a print-out 
of this information is produced to aid shop managers in 
projecting the ratesand mix of work that will be entering 
their respective shops in the near future. 

The inauguration of the WIPICS prototype was accomplished 
by creating computer records for all jobs presently in the 
shop to preclude a lengthy start up before evaluation could 
begin. Simultaneously, the operational costs accountable to 
WIPICS were transfered as overhead to be applied to the price 
of jobs in shop. Thus, during January, 1972, NARFNI initiated 


Seteennologrealizchange which was to be subjected to 
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cost-effectiveness analysis and, at the same time, provided 


aMfsuvtab ler situation to use in investigating a method for 


auditing cost-effectiveness analyses. 


PS COPE OF STUDY 


Brevrousswork eondueted by Spooner, Bradley, and Trefton 


was directed at completing a methodology for determining 


ıf NARFNI is operating more efficiently after installation 


of the WIPICS prototype. However, the method is relatively 


new itself and numerous problems prevented completion of 


the effort. One major problem was the lack of production 


data after the incorporation of WIPICS. The purpose of this 


study is to complete that work. The remaining portions of 


the study are divided into the following catagories: 


le 


Discussion of the methodology and production data 
available from NARFNI. 


Preliminary estimation of NARF costs as functions 
of the work load and prices of inputs, and analysis 
of the characteristics of these functions. 


Estimation and analysis of before and after WIPICS 
cost functions followed by a direct comparison of the 


LW cotcib SJ bU EIBOBKN 


Discussion of the exogeneous variables which were 


a e ludedewrnethel?estuimatwomn of the cost functions. 


Conclusions. 
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¡ME TIOBSTOSYZAND. DATA 


A. GENERAL 

The instructional objective is to determine the effective- 
ness of WIPICS. The major difficulty in attaining this 
objective is the probable abundance of other changes which 
make the before and after WIPICS situations different. Thus, 
a direct comparison of costs is invalid. The ideal solution 
would be to operate two 'tidentical'' NARF's, one with and one 
without WIPICS, and then compare costs., While this solution 
is unobtainable an estimation of its outcome is not. The 
costs of those two NARFs can be estimated through the use of 
econometric techniques to construct before and after WIPICS 
cost functions. The estimation methods were reported by 
Trafton but are outlined in Sections B, C, and D for reference. 

On de to estimate the cost sunctions#ftheir structure 
must. first be specified by solving a mathematical program in 
which NARFNI is assumed to be minimizing costs subject to a 


production function, 


¡E A STS 


minimi 
11013110031262 C 1^1 n^n 


(1) 
subject to Y = F(X4,..X ). 
As stated in Chapter I, NARFNI has three major programs, 


many different outputs. For example, the‏ 010 وٹ ٹ۰ 


airframe program produces airframe repairs on 20 quite 
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different aircraft, and the engine program has an even more 
varied output. To obtain a more tractable situation the 
problem at (1) was solved and then applied to the major 

NARFNI programs separately. The result is one cost function 
Ce he airframe program and one for the engine program. 
Before describing the form of the cost functions a description 
of the data available and the types of production functions 


assumed to hold is necessary. 


B. NARFNI PRODUCTION DATA 

The data was obtained from NARFNI's Production Performance 
Report, a quarterly report which details the total resources 
expended on each airframe and engine which has completed its 
repair process. The report does not include detailed infor- 
mation on the component parts program but rather a monthly 
Summary of total resources expended in that program. A 
description of the data for airframes and engines is given 
below. 

Ir. Identification - includes the type of aircraft or 
engine and the extent of work required, such as 
overhaul, repair of crash damage, modification, or 
conversion. The type of work required affects the 
NORM assigned to the item. 

2. NORM - is NARFNI's estimate of the direct labor hours 


required to accomplish the work. NORM is established 
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at the Quarterly Planning Conference before in- 
duction ofthe item. 

3. Induction and completion date - are the Julian dates 
the item actually entered rework and then work 
completed. 

Di redt labor hours E is the total labor hours 
that can be cifically identified with rework of 
the item. 

5% Direct labor dollars (LS) - iS the total cost of 
the direct labor expended on the item. 

6. Material dollars (M$) - is the total cost of materials 
that can be specifically identified with rework of 
the item. 

7. Overhead dollars (OS) = is a prorated portion of 
those NARFNI costs which cannot be identified with 
specific items in rework. 

The data taken from the Production Performance Reports 

Are raw in the sense that they represent actual costs per 
item. Bradley wrote a computer program which prorates the 
NORM, DLH, DL$, M, O for each item over the number of days 
See SOPA Thesprogram then aggregates. the pro- 
rated data to obtain Ao Ja ٠ت سک رر 6ا٣ تاد‎ airframe 
programs for the period under study. Bradley's program 
generates one additional variable, the number of items in 
rework (I) on each day. Trafton modified these programs 


6 
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IE a ea al a CoS (FES) chargeable to each item for 
every day it was in the NARF ん p. ہے‎ This was an effort 
to reflect pipeline costs which may be the costs most sig- 
nificantly altered by WIPICS. These penalty costs are 
especially significant in the component parts program. A 
relatively small part, such as’a generator, may be the prime 
reason an F-4 aircraft is non-operationäl at its home base. 
Unfortunately, lack of data in the component parts program 
prevented inclusion of that program in this thesis. 

Two variables were constructed from the aggregated data. 


PL = 5 


= , which is a rough surrogate for the price 
L cC lI EE 





I2 


TeS which is an average penalty cost per item 
L on a given day. 


Notice that PL will vary with changes in the amount of 
overtime employed as well as a general pay raise. It may 
also vary as the distribution of jobs in the work load 


changes from one level of skill to another. 


er NESCHON FUNCTIONS 

Two structures for the production function in the economic 
problem of Section A were used, the Cobb-Douglas and the 
۰ئ‎ ٣٠٠۹٠٠۰۳۳۷٣۰۹٣٣٣ cl UDsStitum T CES) "Production runc- 
tions. This was done to compare the ability of each to 
model the relationship between NARFNI's inputs and outputs. 


In each program the output is approximated by NORM produced 
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each day. This approximation is necessary due to the numerous 
types of airframes and engines being repaired. Since NORM 
is measured in the same units for every item it is a simple 
way of measuring production while at the same time reflecting 
the amount of work accomplished by NARFNI each day. 

The Cobb-Douglas production function was assumed to have 
three inputs: 

IES. th price PL 

zo Wi TO price Pl 

3. M with price 1 (since material is measured in dollars). 
Thus the production function takes on the form 


ED cunt c 7۶۳ (2) 


3 
The CES prođuction function was assumed to have two inputs, 
Lend I, and takes the form 
-% 
Pe 
N = YylgITé+ (1-9) L | 
or InN = f cun jr (1-8) p (3) 
The Cobb-Douglas production function was selected for its 
useful and convenient application in emperial work. It is 
gop pecol Case Gi Che CES function in that the limit of the 
rodar eto Tunc teon as ¢ oO. 1S tie Cobb-Douglas function. 
٣٢٣٣٢ ٥٥٦ ح۰٦٥١6‎ elasticity of substitution (6) of inputs is 


assumed fixed at one in the Cobb-Douglas function and it is 


only assumed to be a constant in the CES function Vee p. 2 8 
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The CES production function was included in the study 
specifically because it allows ٣9۰۰۱٦۹ 7 7 POM Win tbs and 
therefore, should present a more accurate model of NARFNI's 
production relation should this be the case. This advantage 
Msc however, by the fact that the CES function at (3) 
above is not linear in the parameters and estimation of the 


resulting cost function becomes more difficult. 


DES TEMATING TAE COST FONCIIONS 

The following procedures apply to both the airframe and 
SiGe COS function estimation. ror this reason only the 
airframe procedures are presented. 

IM COBb-DougTas”"Cost”Eunction 


The cost function is derived by solving 


II 


mine ھ۶‎ 1 TMT PTE) 
PS 
ARI /see Oo D 32/ 


where the dependent variable in the objective function is 


St. N 


defined as DLS + M+ P. The Lagrangian (LG) is 
E apo 
pol prO n HM NAP I MOM (4) 


The 1st order conditions are obtained by taking partial 


derivatives of (4) with respect to L, M, I, and . The results 
are: 
Co ا٣‎ 
LG; 2 PL *AKAL IMA (5) 
ex ا‎ 
po بب,ؤبؤ ؤؤؤ‎ a m (6) 
5 x_B-1 Y 
LG: - PI t ABAL'I M Mj (7) 
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LG, = ×-۸ Ême (8) 
where the 's are random variables associated with errors 
due to imperfect cost minimization by NARENI. 
inese@letzorder conditions cau be solved to obtain 
tae ıimputsäls#runetions#or Thefprices and output,  Substitu- 


tion of these functions into the logarithmic form of the 


eostalunetlionsyıelds 


B 


InC = a + InN + inPL + KF TF 


gry Sr InPI +€ (9) 
where the error term (€) is assumed to be normally distri- 
buted with mean zero and constant variance. 

Standard linear least-squares regression techniques 
/5; Chaps were used to estimate the coefficientS of the 
variables in (9). The results are reported in Chapters III 


and JV: 


eons temuse lasticulys of SubDSt1 tutmon Cost ک7‎ ۷۹ ٦ 
Again M Ee 2٦٠٦1 بے‎ ۲3.6010۱ obtained by Solving 
min C = PT (1 ) ad) 
st N = 2 rO) Ee 


where C is now defined as DLS + O + P. The resulting function 


is lnC 2«-* In pr + eA) | nr arm ۶ (10) 


in which 


A = - کت‎ 

EE (11) 
=e 

he 

DE in 10-9 + T1 





Merc detona O) nor linear 1n the Original 
parameters of the production function and, therefore, cannot 
be estimated by the same procedures useđ on the Cobb-Douglas 
cost function. A more complex two-stage least-squares re- 
gression technique was used instead. The first stage comsists 


of a linear least-squares estimation of 


Im a A رر‎ (12) 
where 
=. 1 
4 = ریچ‎ e 


Equation (12) is derived from the first order ئ91‎ ۹٣ 
the mathematical program /see ira 7٦ 18/. The result- 
ing estimate of 03 is then substituted into D at (11). 
The estimate of D is then substituted into (10) which becomes 
0 in the parameters. Thus, the second stage consists 
of a linear least-squares regression to estimate the co- 
efficients of (10). 

A comparison of the Cobb-Douglas and CES cost functions 
derived is discussed in Chapter III where a preliminary 
analysis of the procedure is reported prior to application 


of the entire auditing methodology. 
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ERBE PDTRECTZEOMPARTSON OF THE COST FUNCTIONS 

To estimate the effect of WIPICS on NARFNI's efficiency 
the cost functions derived in Section D were estimated using 
aggregated production data from a period of time prior to the 
inauguration Of WIPICS to obtain before-WIPICS engine and 
airframe cost functions. The procedure was then repeated 
using aggregated production data for a Nene a Tt ar Wipes 


oran after WIPIECS cost function for each program (Fig..4). 


Before WIPICS After WIPICS 






















O ames GD cost function | CD cost function 
CES cost function | CES Cost tI | 
| 
Engines CD COS EL LUDNCLI ON | CD cost 37:007 


i | | 
CESZeest function CES COST مج‎ m 


HUA ost Functions Estimated 


To compare the two situations the before and after data 
were consolidated and used in the before and after cost func- 
tions to estimate daily program costs for a given period. By 
averaging the difference between the expected cost with 
WIPICS and the expected cost without WIPICS a mean cost 
men Ca 1S predicted tor eachfof the following situations: 


1. NARFNI operation with and without'WIPICS using before 
WIPICS observations. 


2. NARFNI operation with and without WIPICS using after 
WIPICS observations. 


Ze 
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This procedure is followed for both the Cobb-Douglas and CES 
functions. The computer program constructed for this purpose 


is listed at Appendix A. 


F. DATA 

Production data was obtained on the engine program for 
two separate periods. The first period covered 1865 engines 
Indueted omssJulıan dates ON40 through 127329 The adaregated 
results obtained from this period is listed in Appendix B 
of Bradley's thesis. The second period covered 521 engines 
inducted on 2003 through 3050 with the aggregated results 
listed at Appendix B of this thesis. Data on the eer ane 
program covered 564 airframes inducted on 0069 through 2234. 
The aggregated results from Julian date 1001 forward are 
listed in Appendix C. 

The operational cost of WIPICS chargeable to NARFNI 
averages approximately $48,000 per month or $1600 per day. 
NARFNI was not held accountable for these charges until 


January, 1973. 
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10001 و‎ ANALYSIS 


Before attempting to apply the procedures outlined in 
Chapter II a preliminary analysis was performed to develop 
methods of characterizing the estimation techniques. This 
was accomplished by using a 400 day period of data to esti- 
mate cost functions and test for stability and accuracy. 
The resulting proceedures were then included in the final 
analysis of the technological change. The 400 day period 
used in this chapter was well before the WIPICS prototype 
became operational. Therefore, the results obtained should 
reveal some of the problems to look for in the estimation 
of the cost functions, that is, the expected variability of 
the cost function due to events other than the implementation 


om WIPICS. 


A. REGRESSION RESULTS 

Using e procedures outlined in Chapter II Cobb-Douglas 
and CES cost functions were estimated from 300 days of 
aggregated P Bncwrocmbt!nescoetiseroenbsssre 
presented in Table I. The results of the first stage of the 
estimation procedure for the CES cost function is not pre- 
sented unless the coefficients are not significantly different 
From 7 rs when multiplied by 100, is the percentage of 


total variation in cost that can be explained by the estimated 


24 





2) 


uno tone 


Sois tandard error of the estimated cost 


function (in logarithmic form), and the number in parenthesis 


is the standard error of the coefficient just above it. 


Standard errors for leading coefficients are not computed. 











e oe. ere cool ———— S 

| Function Coefficients 5 SE æ 
| | 
| cp; .624 MOON TTS 7 20003 
| سس‎ oO) 

。 1 | 

Airframes | 
CES > 0,870 -.2.461 980 | 
| ( .007) (.182) | 
| — | 
و‎ 3.046 .877 zu .032 | .035 907 | 
| (.017920.095) 70052) | 
| | 

Engines” | | | 
) ٣ ۹ ک8‎ + , > NEUES 808 
| (.025) . 0 | 
L EM 

TABLE I. Before WIPICS Cost Functions (Preliminary) 
1 


EO obServatiolhSs trom OIOTY to 1220. 


2 Beomssbservatıons Geom O215 to. 1150 
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It should be noted that the dependent variable of the 


CES second stage regression is not lnC but rather 


C 
— 
ne PLEPI A 


Also, the C representing costs in the Cobb-Douglas function 
is the sum of material, penalty, and labor dollars, where as 
the C in the CES regression is overhead, penalty, and labor 
dollars. These two differences may explain why, in the air- 
frame program, the CES function has a higher R^ than the 
Cobb-Douglas (indicating a better fit) while the Cobb-Douglas 
has a lower standard error (indicating it is more accurate). 
The two functions are thus not comparable in this form so 
different methods of comparison were needed. 

A rough test for the significance of the coefficeints 
table" "can be made by taking the ration of the coefficient 
to its standard error. If this ratio is less than two the 
Coefficient 12s not signifi antiy different from zero at 
approximately the 5% level of significance. All coefficients 
۰۰۰۔٦١۹2‎ arCeSignittcantiy different from zero by this 
test and the conclusion can be made that each independent 
varıable contributes to the explanation of the dependent 


Variable 
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E EIS OF THE MODELS (CHOW TEST) 

With the large fluxuation in work load and type at NARFNI 
(see NORM and penalty cost values in Appendices B and C) it 
1S probable that the production and cost functions for NARFNI's 
programs shift considerably even though technology is in 
fact not changing. Inherent instability such as this may 
camouflage the shift due to technology and should, therefore, 
be tested for by means of the Chow Test /3, pp. 104-112/. 

The Chow Test is founded on the general principle that a re- 
gression line will not pass through every data point and that 
data obtained from two different populations can be better 
fit with a regression for each population than with a single 
regression Lor ME total data setz If the converse of the 
above statement 1s assumed to hold, then a test to see if two 
sets of data are from the same population may be obtained by 
seeing if a single regression fits as well as two separate 
regressions on the respective data sets. To extend this test 
to the cost functions a second regression was performed on 
the 100 حم کات‎ following the 300 days in each of the 

two programs under study. The data was then combined and 
regressions run on the 400 day periods. The test statistic 
used was 


n4(SE,) = N3(SE3)-n,(SE,) ee ole 


EE ns ee v E 
n„(SE,) + n4 (SE 2k 


1) 


2 





وس 


The subscripts refer to the respective data periods of 400, 
O OO day u ser ise thesresulting equation'sästandard 
error, n is the number of observations, and K is the number 
of coefficients estimated in each equation. If the test 
statistic ls too large then the hypothesis that the two 
periods have the same cost function is rejected. Results are 


presented in Table II. 


Test 
Model Slatistic RN ritica Value Results 






eee 











| F(4,392) | 

Airframes | کت‎ el 
F(5,390) | 

2 | 2.2/7 | T 

| F(4,392) | | 

Engines 14.741 | 2.43 PRESI 

BEES 350) | | 

CES | 41.22 | 2027-۲ | Reject | 


TABLE II. Chow Test Results (Preliminary) 


The entire 400 day periods used in the Chow Test fell. 
well before WIPICS was operational at NARFNI indicating the 
cost ty c are not stable, even with fixed technology. 
This revealed a possible problem in applying the auditing 
methodology proposed by Spooner. However, it may be possible 
Tas dımenısh the importance Of the instability if it is not 
too great. This can be done by increasing the sample size to 


well over 400 days, thereby allowing the variations in work 


20 





Si: | 


load to average out. This technique would prove ineffective 
against large, prolonged projects of a particular work type 
such als major modification of all F24®aircraft in the Navy's 
Inventors. im addition, this)selution would mean waiting, 
perhaps for years, before obtaining enough data, after the 
peechnotogrcesd change; to conduct the analysis. In most cases 
the requirement for the study would have long since been 
0۶ ٤ 

A more generally acceptable solution ala be to accept 
the cost function for use as long as it can accurately pre- 
dict costs while at the same time keeping the accurracy in 
mind when final conclusions are drawn. This acceptance can 
only be justified If the accuracies of the function are 


measured. 


See oCCURACY OF THE COST FUNCTIONS i 
im Empirical Confidence intervals 
- . . €À 

9 e; 1s tee TONO nen ed Cred Cost Lor tne | 
observation a confidence interval can be empiricaly constructed 
Brzusimg the ratio of two Standard errors? of the function 
over the actual mean costs for the period. The result is a 
percent error in cost, 


Ihe term “sittandard error’ is miss-used here since 
potato nta bat the distrabutiom of “ehe error téxms. 
Still, the procedure is not uncommon. 
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PROOI = BETT سش‎ 2100, 





The % error approximates + 2 standard deviations which 
accounts for about 90% of the mass in most probability 
m Stssbutrons s Thus, there is approximately a 9096 confidence 
that the predicted cost is within the percent error com- 
muced Of the actuall cost. 

This procedure was performed on the cost functions esti- 
mated from the 300 day period, and then on the 100 day period 
with costs being predicted by the 300 day models (Table III). 


Model 300 Observatias 100 Observations 
04.0% + 05.0% | 









BEurrtrames 





Engines 
TIE TINS Errors in Cos Functions (Preliminary) 


As the standard errors seemed to indicate in Table I, 
the CES cost function appears better for the airframe pio- 
gram while the results are mixed for the engine program. 
The largest errors are in the engine program when the functions 
vee ood eLGepredict costs, Tos ha sfunderstandable since 
the engine work load and mix of work fluctuate more rapidly 
and extremely than those of the airframe program. The re- 


eunits alsosindicate that the engine cost functions may be 
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highly sensitive to small shifts in the data period used. 
Finally, in the 300 day functions for engines, the Cobb- 
Douglas model appears to do a better Job than the CES. This 
is counter to expected results and is not explained. 
بح ہت تہ‎ ٣ 7 Intervals 

AR sima Max procedure to that of subsection 1 uses 
statistical methods outlined in Theil /pp - 134-135/ to con- 
struct prediction intervals. Let C. represent the estimated 
cost and accept the assumptions of the standard gJanear mode) 
/ Pp: A/A Then the following probability statement 
ho Es 


perc sie ایک‎ ۰-646 


where 
| o d 
H = Cay (SE) 1+V, (X X) "Vo 
X = matrix of independent variablesused to construct 
the model 
Vo = vector of independent variables used to predict 
lnCj 


er { -statistic 
This interval is in logarithms. The transformed interval 
in costs becomes non-symetric and non-linear in the inde- 
pendent variable of Vo. Thus, it is more accurate to compare 
the extreme widths of the intervals than the mean values 


compared in subsection 1l. 
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Prediction intervals were computed for the 100 day 
pomuodssubxpgscbes300-dav models of Table I. Of the 100 
prediction intervals resulting the minimum and maximum in- 
terval were transformed to dollars and re by the 
mean cost for the 100 day period to obtain a maximum and 
minimum width as a percentage of the average cost (parenthe- 
tical entries of Table IV). Finally, the percentage of actual 
costs that fell outside their corresponding prediction 
interval is reported in the last column of Table IV. Com- 
puter programs written to perform these computations are 


reproduced at Appendices D and E. i 
Max. bg % Out- 











Model Interval ln eva sadesinterval 
5305 | 4% 
Airframes CD 5 (6.8%) 


Engines | CD 


| CES (Zor 


TABLE IV. Extreme Widths of 90% Prediction 
Intervals (in dollars) 


Sree Curl 000011007 6 1-72 
Table III and IV should be roughly comparable by 
doubling the % error under the 300 observations column of 
Table III. This value should fall between the maximum and 


minimum percentage values for the same function in Table IV. 
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Asiaan befseen, the statistical values of Table IV tend to be 
smaller when this comparison is made. Moreover, a larger 
discrepancy between the two tables is accompanied by a larger 
percentage of Lhepactual costs falling outside of the pre- 

.٠3 ت٣3‎ 0 interval or lable IV. 

Some of this difference may be accounted for as 
enterpolation errors for the t -statistic used in the equa- 
LIE ma xob»tainnelablespnv. Errors zn ithisfstatistic are 
exponential in nature and the resulting emm intervals 
were found to be quite sensitive to the value used. The same 

-statistic value was used for all of the functions in 
Table IV which seems to imply the value chosen was low. A 
slight increase in this value would increase the prediction 
intervals, make Table IV more consistent with Table III, 
and should, at the same time, decrease the percentage of 
actual costs falling outside the corresponding prediction 
interval. More care was taken in selecting this value in 
Chapter IV where the auditing methodology is carried through 
completely. 

Finally, the uneven prediction results between the 
cost functions off¥lable IU may Be indicative of th Chow 
Mas iS ACONGIUSION that thestwo periods o data, 300 vs. the 
Bollowincg O0 QJays, are Intraect different andia model of one 


E rC ISRA WOOT prediction of costs in another period. This 
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possibility is important because the periods are not separated 
by a recognizable technological change. It reemphasizes the 
need to include the accuracy of the models in the final com- 


parison of estimated costs. 
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IV. COMPARISON OF BEFORE AND AFTER 


' WIPICS PREDICTED COSTS 


THE COST FUNCTIONS 

The final cost functions used to compare estimated costs 
across the technological change were obtained using the 
SNAP/IEDA Computing Package developed by the Department of 
Statistics, Princeton University. SNAP/IEDA will accept 
up to 400 observations. In those models requiring EO 400 
observations the BIMED-O2R program, developed by the Health 
Sciences Computing Facility, University of California, was 
used for the Cobb-Douglas estimation. The CES estimation 
was limited to 400 observations in all cases since estima- 
tion using the BIMED-O2R program would become a 3 stage 
procedure requiring excessive manipulation and,inefficient 
Se GCI CMU EE (times. Also, for ine final cost functions, 
the data periods were altered for two reasons. First, the 
eee ods were chosen close to the technological change 
so the before and after situations would be as similar as 
possible, except in technology. Second, the data periods 
were altered in size and content to reduce the effects of 
instability as much as possible. The results are presented 


LO bales ey wand Vi Withetootnotes indicating the data used. 





= 


Note that the CES function for engines across WIPICS 
was not due to the large number of observations 
involved. Omission of this function merely precludes per- 
forming a Chow test which in all probability would be rejected 
if performed. Visual comparison of the before and after 


functions supports this. 


{TOO MEF SR ee Se es eee ےہ‎ 








Function Coefficients SE 
| 


i 


CD ee ODT 22 "۳6 -.094 .014 

















Before MEDIAS) (.090) OS ) 
WIPICS | 
2.999 . 886 10-221 ‚025 
| (MOTO ‚543 
i 
-1 213 .745 4.222 2,487 ‚017 
After | (2071) LS OS CI 半 | 
WIPICS 
2.008 .881 2 8 .OO6 
(-OTO mE - - OSA ーー 
.934 .884 ‚674 8 Zo 1020 
Acrossg oe OSS (7023) 
WIPICS 
‚945 | .907 و‎ ‚031 E 
025) 000 
TABLE: VSR ٥۹٢2701 6 3ے .کے0 6و‎ CONS 
4 


400 observations from 0316 to 1350. 


> 89۰567 676006 ھو11120007٦٦9‎ 170 


e خی گ۰‎ 2 above excepi tes "ending in 3, 5, or 7 
Oe To TENCE Lhe mumber mwmemebservations to 343. This tech- 
Duqueswasmapplred to the before andygafter case with essentially 
Mnerseme Mene AI cn tStesultine. as reported above. Thus, de- 
letrtonfor some data should not have significantly affected the 
Serossi WIEICS coeficientes. E 
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463 observations 


400 observations 


9 160) observations 


10 K 
623 observatıons 


Function Coefficients 
Before E | 2.104 E: .147* 
WIPICS ; 08۹ NE C70 
css? 1.357 . 7 -.135% 
| | (.026) (.422) 
"After y ¡CD { 4.021 . 708 - .750 
WIPICS | (.020) (.163)  (.080) 
Merc S2 4 Tai 
| | COS) ee, 
een Ell 
| | 
| 
Across, ,|CD 0 3,408 229 . 345 271 
WIPICS “} (OR ( .043) (OS) 
i { 
i, 
٢٢٢٠١ +١ 7 
EXAM uu t 
TABLE VI Nine osii kuncett rons 
* Not significant at the 5% level. 
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All coefficients for the airframe functions are signi- 
facantly different from zero. In addition to the two starred 
coefficients which were not significant in the engine functions, 
ibezcoeffieent Of PH e ne rs Berge CI CME Deron. 
DC ORC ES estimation was not significant. Since the 
functions were estimated from varying time periods the SE 
and R^ are representative of only the Ferne AS function and 
should not be compared across functions. Thus, to characterize 
the functions the Chow Test and statistical prediction inter- 


val discussed in Chapter III were used. 


STABILITY ACROSS THE CHANGE‏ و 
iMiemGhew lesızswas used on the cost functions to determine‏ 
if they were in fact different before and after WIPICS‏ 


labile Vil. 










ab Col Results 
Model 0۷011 - 6 Value 
Airframes CD 16.94 (4.362) Reject 
t 2.43 cs sa 
CES (2,202) Reject 
2.43 p 
CD (4,615) Reject 
Engines 2.43 


(Not computed) 


ューーー- nme e مس‎ 


TABLES ID Chow lest Results 


ES 
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In all tests the hypothesis that the before and after WIPICS's 
cost functions are the same was soundly rejected. This im- 
plies the additional conclusion that a technological change 
tmay't have affected the relationship between imputs and out- 


puts. 


CS HREDICTION INTERVALS 

90% prediction intervals were calculated for the before 
and after WIPICS's functions by using the functions to pre- 
dict costs for the last 100 days (90 days for after WIPICS 
airframe functions) of this respective time periods. The 
same procedure used to construct Table IV was then used T 
Bonstruct Table VIII, 

The most striking result is the improvement obtained in 
percent of observations falling outside the prediction in- 
terval. This occurred because the 100 costs predicted were 
EN ub.ctocrcrthemsanmpaesused ctoseconstruct the cost 
functions. A second difference from Table IV is the size 
of the prediction intervals. The change was mixed, some 
larger and some smaller. This again points to the relatively 
unstable situation at NARFNI in which prices and amounts of 
inputs appear to be varying widely. 

Although the prediction intervals are non-symetric about 
pue predgwscled cost? onefhalf or the maximum interval should 


Safely represent at least two standard errors of the cost 
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Eine lito value wiii be uses tó make a significance 
statement about the cost comparison of operating with and 


without WIPICS. 


CO RECI COMPARISON ORF PREDICTED COSTS 
Following the comparison procedures discussed in Section 
E, Chapter II, the cost differentials in Tables IX and X 


"Wero mel L ained. 





CD CES 
BW-AW) (BW-AW) 
Before WIPICS S17885 S 3306 
Para 
After WIPICS $9241 $7129 


Data 


TABLE IX. Difference Between Predicted 
GoctessAirframes 





li Positive value indicates operation with WIPICS is 
estimated to be cheaper. 
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CES 


(AW-AW) 





Before WIPICS S 5286 
Data 
Aitor WIPICS $3893 
Data 
TABLE X. Difference Between Predicted 
Costsl^ - Engines 
Table IX is read as saying, "Based on the before and 


after WIPICS Cobb-Douglas cost functions it is estimated 

that WIPICS could have saved NARFNI $17,885 per day in the 

er: frame, program during the period 0316 to 1350, had WPICS 
Berememroperation at thal time." Notice that this says nothing 
about changes which these cost کت‎ cannot measure, 

such as an arbitrary change in the NORMS assigned to in 

coming repair items. These factors will be B ed in 
Chapter V. In addition, the statement does not consider 

the inaccuracies of the models which were measured in Table 

EU IUE t should be qualified by the assumption that the 


Faıfference is not significant unless it exceeds, in absolute 


value, the sum of one-half of the two maximum prediction 


mage ee‏ ر ee‏ ےت سے 


12 Positive value indicates operation with WIPICS is 


estimated to be cheaper. 
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intervals of the cost functions involved en the above ex- 
ample this sum is $4776 (computed from Table STATS EEUU S 
the statement is significant with at least 90% confidence 

ER disregarding externalities to be discussed 70 ۹۹۶۹۶۹ ٣٦ 
V]. 

SING IM be pProcequt e to interpret Tables IX and X 
the CD models indicate a significant advantage in the air- 
frame program and an insignificant disadvantage in the engine 
program. The CES models indicate an insignificant WIPICS 
advantage in both programs. 

The above comparison could be refined by removing the 
effects of extlernalities. This: should reduce the total 
variance of costs and it should reduce the maximum predic- 
tion intervals of the cost functions. The net result would 
then be a smaller region in which the cost differential would 
be considered insignificant, but also possibly smaller in- 


dieatedWeost savings. 
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٠۰۰ ۱۱۰۱ 11.۳ 2ج‎ EXOGENEOUS TO WIPICS 


en re نے نے سے کا‎ a a a NE ہے‎ 


The Cobb-Douglas and CES models do an imperfect job of 
Petia ic tie er rece of WEPICS en production costs due to 
many changes within the periods which were not caused by 
WIPICS. If the major changes were isolated, and the data 
corrected to account for them, then the proceedures discussed 
in this thesis should give an acceptable estimation of WIPICS! 


effect. Some of these changes are noted below. 


eer RICE CHANGES 

Pook ae minimization prooiem Of Chapter Il eam 
increase in the price of an input would result in higher 
GostsS lomproducesasgiven level of output. Alternatively, 
a cost function estimated from data with low prices would 
tend to underestimate the cost of production for data taken 
when prices are high. For this reason increasing prices would 
depreciate the ۶ت ہت‎ ٣۲6۴۰٠٠٠٠۶٠٠٠٠٥6 7 716 3-7 6113 96 The 
price of et labor and penalty costs are accounted for in 
E تب٣٠‎ ٦ت‎ ۲3100 6-1166 for this study so their fluctuations 
rDoukcemerseiteet the Linal compartson OF predicted costs. 
This is not the case for material prices and the prices of 
indirect labor absorbed into overhead. Shifts in these 
latter prices could, and should, be corrected by the appli- 


Cal LOmmen var LCemIndex, 
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B. COMMAND POLICY CHANGES 

ea ressıs arten elusive and difficult to identify with 
specific actions. However, obvious changes, such as a new 
ceiling on allowable overtime, should be accounted for. 
Dgsucrcase im Overtime, If not a result of command policy 
changes, may be caused by temporary Ines tenor 7۲ 
conjestion due to the learning process that often follows a 
reehnologieal change, In tbis case the correction would 
consist of identifying the period affected and deleting it 
73۲۰" ) 6-7 

Another example is a shift in the standards for assign- 
ing NORMS at NARFNI. The shift is acceptable if experience 
shows that required labor hours have actually changed. Any 
other reason for the shift would result in an imaginary shift 
in the production function with a concurrent change in the 
drived cost function. This type of bias could also be 
corrected by application of an index to the data before 


estimating the cost functions. 


C. PERSONNEL TURNOVER 

A season of large personnel cuts followed by hiring of 
new personnel would naturally establish a training period 
OREN Ls Sburing thas period the relative efficiency of 
workers would be low and a corresponding change in the cost 


Due lonswonld bce Motel. Ir the training occurred during a 
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period used to estimate a ES Lumet ton Cie function would 
finer plas resmiismg m comparison errors. The effect 

is complex, however, it may be that wages paid to personnel 
are low enough, relatively speaking, to allow the cost of 

PM oduction- at a given level to remain approximately constant. 
Still, the relation between the 75ص‎ be altered and 


a detailed study would be required to correct con tne (plas 


D. CHANGES IN TYPE OF WORK REQUIRED 

In Chapter I it was explained that NARFNI has many different 
Gutputs in the form of repairs to various types of alrframes 
and engines. These numerous Suntpele were aggregated into 
only two products, repairs of airframes and engines. The 
cost functions associated with these two products should do 
an acceptable job of predicting costs so long as the misol 
work remains fairly constant within the respective programs. 
A shift in work type cannot be accounted for by the cost 
functions. Unfortunately, there is strong evidence that the 
work loads are shifting considerably. An approach to ac Count 
ing for these shifts is discussed in Section C of Chapter VI, 


PROBLEMS OF FURTHER RESEARCH. 
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VI. CONCLUSIONS AND PROBLEMS OF FURTHER RESEARCH 


موی نے me eee ce‏ ا re re‏ سوہ 


DEN RRBECSTVENBESS OR WIPICS 

The comparison of estimated costs for the "with" and 
Mwithout'" WIPICS situations indicated a savings in the air- 
Ane np O gral ۳۰7۱۶ ۳٣۱۳۳۱7۳۳ بت 0703 11ت‎ 0 Ihe results 
of the two program comparisons are not additive due to the 
difference in time periods considered in each program. In 
addition, the component parts program was excluded fron the 
study because of data format. For these reasons, an overall 
eonelusion concerning the cost-effectiveness of WIPICS Ni 
nısbegextracted from this study. 

Upon completion of the study it was found that WIPICS 
had not been fully operational in the engine and airframe 
programs for the periods originally stated. Numerous 
mol one in implementing the system prevented a complete test 
EE NeEÉsystem during this period: IWIPICS has been completely 
operational in the airframe program since October of 1972 
but was not operational in the engine program as of 4 May, 1973. 

0“ 111177 0م"‎ savings in the airframe 
program may be explained by shifts in the external variables 
Ansel ssed Sat ap rer a GC rime = sispect being changes in the 


types of work. 
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Beau TANG METHODOLOGY 


Investigation of the auditing methodology originally 


proposed by Spooner revealed eight primary steps that are 


Esaential to arriving at a strong conclusion: 


t. 


T: 


er 


Identify external variations such as command policies 
and inflation of prices so they may be accounted for 
in the data. 

Corre ei the data for the variations identified in 

e Si tudat rono beforeandafter the‏ وہ لا 
technological change will be as similar as possible.‏ 
Aggregate the data.‏ 

Select the periods to study, keeping in mind that 
closeness to the change will aid in stabilizing 
functions while, at the same time, a delay may be 
needed to allow the technological change to become 
noticeable. 

Estimate the cCoStTunctrons. 

Estimate the accurracy of the functions. 

Compare estimated costs derived from the cost functions. 


BPrassconelusıons from oO ana 7 ABOVE. 


The audit conclusions may be more meaningful if identical 


time periods are used in each program studied (assuming there 


1s more than one). This will allow the separate effects to 


I rıves(or zn overall statement about the effectiveness 


ou the change. 
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Both the Cobb-Douglas and CES cost functions appeared to 
Wearyein abidaty EO represent the production relation of inputs 
to outputs. Neither function dominated the other in accuracy 
therefore no conclusion can be made as to which should be 
used. If only one is to be used the natural chodce is بت‎ 
simpler Cobb-Douglas function. 

The results of the WIPICS test case indicate there remain 
problems with the accuracy of the tests. It is believed that 
I 1 Simple treatment oL the data forsexrternal ınrlu- 


ences would improve this accuracy. 


C OBEEMO TOET FURTHER RESEARCH 

There are three primary areas in the auditing ot cost- 
effectiveness analyses undergoing current research at the 
Naver OSteraduate School. 

First, Professor James Hartman 1S applying search tech- 
ao nna tempti to improves the estimation of the 
parameters the CBS cost functions. The 2-stage least- 

E arto regression used in this thesis results in two estimates 
Eo n-aclastuest or aabstirtmbilonH in the production function, 
both of which are significant. The problem is that the two 
estimates are invariably quite different. It is hoped that 
better estimates of this parameter will result in a more 


eeuroteze), cost rimetion Kor use in the auditing methodology. 
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Second, Professor Norman K. Womer is investigating 
methods of accounting for the shifting of work type in the 
various programs which is believed to be responsible for much 
of the inaccuracies in the models used in this thesis. The 
principal effort in this area is toward developing a com- 
puter program to perform the regressions ٠ raw data instead 
of the aggregated data. Dummy variables are being employed, 
one for each type of airframe or engine, to account for 
shifting work type. This requires a number of variables 
exceeding the capabilities of the computer soft-ware presently 
available. 

Finally, Captain Richard McGarrahan, USA, is investigating 
a second auditing methodology proposed by Spooner which 
employs a linear input-output production function instead 
of the continuous production functions assumed in this thesis. 
The objective of the work is to establish a basis by which to 
judge the abilities of the continuous models. The input- 
output model is ےت‎ to be more accurate in that it treats 
each type of work independently and should therefore be more 
Stable. If the continuous models compare favorably in 
accuracy they would then be considered more desirable for 


their ease in handling. 
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